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SPEECH PROCESSING SYSTEM 

The present invention relates to an apparatus for and 

method of speech processing. The invention has 

5 particular, although not exclusive relevance to the 

detection. of speech within an input speech signal. 

=J In some applications, such as speech recognition, speaker 

jil verification and voice transmission systems, the 

li ; 

Vj; 10 microphone used to convert the user's speech into a 

iy corresponding electrical signal is continuously switched 

jU on. Therefore, even when the user is not speaking, there 

1*1 will constantly be an output signal from the microphone 

!=| corresponding to silence or background noise. In order 

! "15 (i) to prevent unnecessary processing of this background 

noise signal; (ii) to prevent misrecognitions caused by 
the noise; and (iii) to increase overall performance, 
such systems employ speech detection circuits which 
continuously monitor the signal from the microphone and 
20 which only activate the main speech processing system 

when speech is identified in the incoming signal. 

Detecting the presence of speech within an input speech 
signal is also necessary for adaptive speech processing 
25 systems which dynamically adjust weights of a filter 
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either during speech or during silence portions. For 
example, in adaptive noise cancellation systems, the 
filter coefficients of the noise filter are only adapted 
when both speech and noise are present. Alternatively 
5 still, in systems which employ adaptive beam forming to 

suppress noise from one or more sources, the beam is only 
adapted when the signal of interest is not present within 
>u the input signal (i.e. during silence periods). In these 

5) systems, it is therefore important to know when the 

ill 

45-0 desired speech to be processed is present within the 

^ input signal. 

□ 
III 

&j Most prior art speech detection circuits detect the 

p beginning and end of speech by monitoring the energy 

5=5: 

15 within the input signal, since during silence the signal 

energy is small but during speech it is large. In 
particular, in conventional systems, speech is detected 
by comparing the average energy with a threshold and 
indicating that speech has started when the average 

20 energy exceeds this threshold. In order for this 

technique to be able to accurately determine the points 
at which speech starts and ends (the so called end 
points), the threshold has to be set near the noise 
floor. This type of system works well in environments 

25 with a low constant level of noise. It is not, however, 
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suitable in many situations where there is a high level 
of noise which can change significantly with time. 
Examples of such situations include in a car, near a 
road or any crowded public place. The noise in these 
5 environments can mask quieter portions of speech and 

changes in the noise level can cause noise to be 
incorrectly detected as speech. 

w 

One aim of the present invention is to provide an 
? jjl0 alternative speech detection system for detecting speech 

iJ - within an input signal. 

ii! 

!i: According to one aspect, the present invention provides 

an apparatus for detecting the presence of speech within 

15 an input audio signal, comprising: a memory for storing 

a probability density function for parameters of a 
predetermined speech model which is assumed to have 
generated a set of received audio signal values; means 
for applying the received set of audio signal values to 

20 the stored probability density function; means for 

processing the probability density function with those 
values applied to obtain values of the parameters that 
are representative of the input audio signal; and means 
for detecting the presence of speech using the obtained 

2 5 parameter values. 
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Exemplary embodiments of the present invention will now 
be described with reference to the accompanying drawings 
in which: 

Figure 1 is a schematic view of a computer which may be 
programmed to operate in accordance with an embodiment of 
the present invention; 



Figure 2 is a block diagram illustrating the principal 
^ components of a speech recognition system which includes 

a speech detection system embodying the present 
invention; 



15 Figure 3 is a block diagram representing a model employed 

by a statistical analysis unit which forms part of the 
speech recognition system shown in Figure 2 ; 



Figure 4 is a flow chart illustrating the processing 
2 0 steps performed by a model order selection unit forming 

part of the statistical analysis unit shown in Figure 2; 
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Figure 5 is a flow chart illustrating the main processing 
steps employed by a Simulation Smoother which forms part 
of the statistical analysis unit shown in Figure 2; 
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Figure 6 is a block diagram illustrating the main 
processing components of the statistical analysis unit 
shown in Figure 2; 

5 Figure 7 is a memory map illustrating the data that is 

stored in a memory which forms part of the statistical 
analysis unit shown in Figure 2; 

Si 

%l Figure 8 is a flow chart illustrating the main processing 

! Jio steps performed by the statistical analysis unit shown in 

i " 

Wi Figure 6; 

til 

jf; Figure 9a is a histogram for a model order of an auto 

: = f regressive filter model which forms part of the model 

15 shown in Figure 3; 

Figure 9b is a histogram for the variance of process 
noise modelled by the model shown in Figure 3; and 

20 Figure 9c is a histogram for a third coefficient of the 

AR filter model. 

Embodiments of the present invention can be implemented 
on computer hardware, but the embodiment to be described 
25 is implemented in software which is run in conjunction 
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with processing hardware such as a personal computer, 
workstation, photocopier, facsimile machine or the like. 



Figure 1 shows a personal computer (PC) 1 which may be 
5 programmed to operate an embodiment of the present 

invention- A keyboard 3, a pointing device 5, a 
microphone 7 and a telephone line 9 are connected to the 
PC 1 via an interface 11- The keyboard 3 and pointing 
]J; device 5 allow the system to be controlled by a user. 

";io The microphone 7 converts the acoustic speech signal of 

the user into an equivalent electrical signal and 
ji; supplies this to the PC 1 for processing. An internal 

:f: modem and speech receiving circuit (not shown) may be 

!=f connected to the telephone line 9 so that the PC 1 can 

15 communicate with, for example, a remote computer or with 

a remote user. 



The program instructions which make the PC 1 operate in 
accordance with the present invention may be supplied for 
20 use with an existing PC 1 on, for example, a storage 

device such as a magnetic disc 13, or by downloading the 
software from the Internet (not shown) via the internal 
modem and telephone line 9. 



25 



The operation of a speech recognition system which 
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employs a speech detection system embodying the present 
invention will now be described with reference to Figure 
2 . Electrical signals representative of the input speech 
from the microphone 7 are input to a filter 15 which 
5 removes unwanted frequencies (in this embodiment 

frequencies above 8 kHz) within the input signal. The 
filtered signal is then sampled (at a rate of 16 kHz) and 

1=1 

;*( digitised by the analogue to digital converter 17 and the 

)i: digitised speech samples are then stored in a buffer 19. 

IJi 
ni 

'13.0 Sequential blocks (or frames) of speech samples are then 

*ii 

s It? 

lJl passed from the buffer 19 to a statistical analysis unit 

v. 

Is* 21 which performs a statistical analysis of each frame of 

:i( speech samples in sequence to determine, amongst other 

H things, a set of auto regressive (AR) coefficients 

15 representative of the speech within the frame. In this 

embodiment, the AR coefficients output by the statistical 
analysis unit 21 are then input to a speech recognition 
unit 25 which compares the AR coefficients for successive 
frames of speech with a set of stored speech models 27, 
20 which may be template based or Hidden Markov Model based, 

to generate a recognition result. In this embodiment, the 
speech recognition unit 25 only performs this speech 
recognition processing when it is enabled to do so by a 
speech detection unit 61 which detects when speech is 
25 present within the input signal. In this way, the speech 
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recognition unit 25 only processes the AR coefficients 
when there is speech within the signal to be recognised. 



In this embodiment, the speech detection unit 61 also 
receives the AR coefficients output by the statistical 
analysis unit 21 together with the AR filter model order, 
which, as will be described below, is also generated by 
the statistical analysis unit 21 and determines from 
these, when speech is present within the signal received 
from the microphone 7. It can do this, since the AR 
filter model order and the AR coefficient values will be 
larger during speech than when there is no speech 
present. Therefore, by comparing the AR filter model 
order and/or the AR coefficient values with appropriate 
threshold values, the speech detection unit 61 can 
determine whether or not speech is present within the 
input signal. 

Statistical Analysis Unit - Theory and Overview 

As mentioned above, the statistical analysis unit 21 
analyses the speech within successive frames of the input 
speech signal. In most speech processing systems, the 
frames are overlapping. However, in this embodiment, the 
frames of speech are non-overlapping and have a duration 
of 20ms which, with the 16kHz sampling rate of the 
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analogue to digital converter 17, results in a frame size 
of 320 samples. 

In order to perform the statistical analysis on each of 
5 the frames, the analysis unit 21 assumes that there is an 

. underlying process which generated each sample within the 
frame. The model of this process used in this embodiment 
is shown in Figure 3. As shown, the process is modelled 
jjj by a speech source 31 which generates, at time t = n, a 

l i%0 raw speech sample s(n). Since there are physical 

: ss 

tJl constraints on the movement of the speech articulators, 

!=f there is some correlation between neighbouring speech 

•ij samples. Therefore, in this embodiment, the speech 

H source 31 is modelled by an auto regressive (AR) process. 

15 In other words, the statistical analysis unit 21 assumes 

that a current raw speech sample (s(n)) can be determined 
from a linear weighted combination of the most recent 
previous raw speech samples, i.e.: 

20 s(ri) = a^ty-l) + a 2 s(n-2) + + a k s(n-k) + e(n) (1) 

where a lf a 2 a k are the AR filter coefficients 

representing the amount of correlation between the speech 
samples? k is the AR filter model order; and e(n) 
25 represents , random process noise which is involved in the 
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generation of the raw speech samples. As those skilled in 
the art of speech processing will appreciate, these AR 
filter coefficients are the same coefficients that the 
linear prediction (LP) analysis estimates albeit using a 
different processing technique. 

As shown in Figure 3, the raw speech samples s(n) 
J{ generated by the speech source are input to a channel 33 

]fj which models the acoustic environment between the speech 

•;10 source 31 and the output of the analogue to digital 
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converter 17. Ideally, the channel 33 should simply 
attenuate the speech as it travels from the source 31 to 
the microphone. However, due to reverberation and other 
distortive effects, the signal (y(n)> output by the 
15 analogue to digital converter 17 will depend not only on 

the current raw speech sample (s(n)) but it will also 
depend upon previous raw speech samples. Therefore, in 
this embodiment, the statistical analysis unit 21 models 
the channel 33 by a moving average (MA) filter, i.e.: 
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y(n) = h Q s(n) + h^sQi-l) + h 2 s(n-2) + + h r s(n-r) + z{n) (2) 



25 



where y(n) represents the signal sample output by the 
analogue to digital converter 17 at time t = n; h 0 , h x , 
h 2 ..-.h r are the channel filter coefficients representing 
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the amount of distortion within the channel 33; r is the 
channel filter model order; and £(n) represents a random 
additive measurement noise component. 



";io 



l si 



For the current frame of speech being processed, the 
filter coefficients for both the speech source and the 
channel are assumed to be constant but unknown . 
Therefore, considering all N samples (where N = 320) in 
the current frame being processed gives: 

s(n) = a x s{n-\) + a 2 s(n-2) + + a k s{n-k) + e(n) 

s(n-\) = a x s{n-2) + a 2 s(n-3) + + a k s(n-k~\) + e(n-\) 



(3) 
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s(n-N+l) = a x s(n-N) + a 2 s(n-N-\) + + a k s(n-k-N+\) + e(n-N+l) 



which can be written in vector form as: 



&(ri) = S.a + e.(ri) 



(4) 
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where 



S = 



s(n-\) s(n-2) s(n-3) 
s(n-2) s(n~3) s(n~4) 
s(n-3) s(n~4) s(n-5) 



s(n-k) 
s(n-k-\) 
s(n~k-2) 



s(n-N) s(n-N-\) s(n~N-2) . . . s(n-k-N+\) 



and 
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a ~ 



£(») - 



kxl 



s{n) 
s{n-\) 
s(n-2) 



e(n) = 



e(n-l) 
e{n-2) 



e(n-N+l) 



Hio 



As will be apparent from the following discussion, it is 
also convenient to rewrite equation (3) in terms of the 
random error component (often referred to as the 
residual) e(n). This gives: 

e(n) = s(n) - a l s(n-\) - a 2 s{n-2) - - a k s(n-k) 

e(n-l) = s(n-l) - a 1 s(n~2) - a 2 s(n-3) - - a k s(n-k-Y) 

(5) 
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c(w-JV+l) = s(n-N+\) - a^sln-N) - a 2 s(n-N-\) - - a k s(n-k-N+\) 

which can be written in vector notation as: 

where 



(6) 
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1 -a } -a 2 -fl 3 
0 1 -a ] -a 2 
0 0 1 -a } 



A = 



0 0 0. 



-a k _ x -a k 0 0. 



0 
0 
0 



AxV 
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Similarly, considering the channel model defined by 
equation (2), with h 0 = 1 (since this provides a more 
stable solution), gives: 

q(n) =h l s(n-\) +h 2 s(n-2) + .. +h r s(n-r) +e(n) 
^(«-l)=/? 1 s(w-2) +h 2 s(n-3) + .. +h r s(n-r-l)+t(n-l) 

(7) 



•tio 



i = \ 
111 
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q(n-N+\) ^h^in-N) +h 2 s(n-N-\) + .. +h r s(n-r-N+l) +e(n-N+l) 

(where q(n) = y(n) - s(n)) which can be written in vector 
form as : 

= r.h + £(«) (8) 

where 



Y = 



s(n-\) s(n-2) s(n-3) 
s(n-2) s(n-3) s(n-4) 
s(n-3) s(n-4) s(n-5) 



s(n-r) 
s(n-r-\) 
s(n-r-2) 



s(n-N) s(n-N-\) s(n-N-2) . . . s(n-r-N+l) 



20 



and 



t = 



ffOO - 



<?(") 

q{n~\) 
q(n-2) 



q(n-X+l) 



e(w-2) 



e(n-A'+l) 



Ail 
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In this embodiment, the analysis unit 21 aims to 
determine, amongst other things, values for the AR filter 
coefficients (a) which best represent the observed signal 
samples (y_(n) ) in the current frame. It does this by 
determining the AR filter coefficients (a) that maximise 
the joint . probability density function of the speech 
model, channel model, speech samples and the noise 
statistics given the observed signal samples output from 
the analogue to digital converter 17, i.e. by 
determining : 



where a B 2 and o E 2 represent the process and measurement 
noise statistics respectively. As those skilled in the 
art will appreciate, this function defines the 
probability that a particular speech model, channel 
model, raw speech samples and noise statistics generated 
the observed frame of speech samples (y(n)) from the 
analogue to digital converter. To do this, the 
statistical analysis unit 21 must determine what this 
function looks like. This problem can be simplified by 
rearranging this probability density function using Bayes 
law to give : 
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p(^(/i)k(n) J A J r J q;)p(^)|a J Ar > q;)p(fl|^)^(A|r)p(^)p(o;)p(A:)p(/-) 



(10) 



if- 

•SXT 

in 
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As those skilled in the art will appreciate, the 
denominator of equation (10) can be ignored since the 
probability of the signals from the analogue to digital 
converter is constant for all choices of model. 
Therefore, the AR filter coefficients that maximise the 
function defined by equation (9) will also maximise the 
numerator of equation (10). 

Each of the terms on the numerator of equation (10) will 
now be considered in turn. 

p(s(n)\a, k, a e 2 ) 

This term represents the joint probability density 
function for generating the vector of raw speech samples 
(s(n)) during a frame, given the AR filter coefficients 
(a), the AR filter model order (k) and the process noise 
statistics (a e 2 ) . From equation (6) above, this joint 
probability density function for the raw speech samples 
can be determined from the joint probability density 
function for the process noise. In particular p(s(n) |a, 
k, a e 2 ) is given by: 
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p(s(.n)\a,k,a e ) = p{e(ri)) 



5e(«) 



8s(«) 



e(w) = s(«) - Sa 



(ID 



iJi 

in 
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where p(e(n) ) is the joint probability density function 
for the process noise during a frame of the input speech 
and the second term on the right-hand side is known as 
the Jacobean of the transformation. In this case, the 
Jacobean is unity because of the triangular form of the 
matrix A (see equations (6) above). 

In this embodiment, the statistical analysis unit 21 
assumes that the process noise associated with the speech 
source 31 is Gaussian having zero mean and some unknown 
variance a e 2 . The statistical analysis unit 21 also 
assumes that the process noise at one time point is 
independent of the process noise at another time point. 
Therefore, the joint probability density function for the 
process noise during a frame of the input speech (which 
defines the probability of any given vector of process 
noise e(n) occurring) is given by: 



p(e{n)) = (2;iaJ 2 exp 



-e(n) T e(n) 



2a 



(12) 
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Therefore, the joint probability density function for a 
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vector of raw speech samples given the AR filter 
coefficients (a), the AR filter model order (k) and the 
process noise variance (a e 2 ) is given by: 



pU(n)\aA<*l) = (27taJ) 2 exp 



-1 
2a? 



[z(n) T z(") - 2a T Ss.{ri) + a T S T Sa) 



(13) 
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This term represents the joint 1 probability density 
function for generating the vector of speech samples 
(y(n)) output from the analogue to digital converter 17, 
given the vector of raw speech samples (s<n)), the 
channel filter coefficients (h) , the channel filter model 
order (r) and the measurement noise statistics (o\ 2 ). 
From equation (8), this joint probability density 
function can be determined from the joint probability 
density function for the process noise. In particular, 
P(y_( n) | s( n) , h, r, o e 2 ) is given by: 
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p(j<w)k(w),4,r 1 o e ) =p(£(w)) 



5fi(/i) 



5*(») 



£(«) = 0(11) - Yh 



(14) 
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where p(i.(n)) is the joint probability density function 
for the measurement noise during a frame of the input 
speech and the second term on the right hand side is the 
Jacobean of the transformation which again has a value of 
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In this embodiment., the statistical analysis unit 21 
assumes that the measurement noise is Gaussian having 
zero mean and some unknown variance o 5 2 . It also assumes 
that the measurement noise at one time point is 
independent of the measurement noise at another time 
point. Therefore, the joint probability density function 
for the measurement noise in a frame of the input speech 
will have the same form as the process noise defined in 
equation (12). Therefore, the joint probability density 
function for a vector of speech samples (y.(n) ) output 
from the analogue to digital converter 17, given the 
channel filter coefficients (h) , the channel filter model 
order (r), the measurement noise statistics (a f 2 ) and the 
raw speech samples ( s.(n) ) will have the following form: 



iV 



pWn)\s!Ln),b.>r,<0 = (2*<) 2 exp 



(a(") r a(«) - 2k T ra(n) + k T r T rk) 



2a: 



(15) 
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As those skilled in the art will appreciate, although 
this joint probability density function for the vector of 
speech samples (y(n)) is in terms of the variable <i(n), 
this does not matter since a(n) is a function of y(n) and 
s_(n), and s(n) is a given variable ( ie known) for this 
probability density function. 
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This term defines the prior probability density function 
for the AR filter coefficients (a) and it allows the 
statistical analysis unit 21 to introduce knowledge about 
what values it expects these coefficients will take. In 
this embodiment, the statistical analysis unit 21 models 
this prior probability density function by a Gaussian 
having an unknown variance (a a 2 ) and mean vector (u*) , 
i.e.: 



_N 

p(a\k,o 2 a ,n) = (2no a ) 2 exp 



(a-u/(a-u o ) 



2ot 



(16) 
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By introducing the new variables o a 2 and y^, the prior 
density functions (p(a a 2 ) and p(Ma) ) for these variables 
must be added to the numerator of equation (10) above. 
Initially, for the first frame of speech being processed 
the mean vector (u a ) can be set to zero and for the 
second and subsequent frames of speech being processed, 
it can be set to the mean vector obtained during the 
processing of the previous frame. In this case, p(£U) is 
just a Dirac delta function located at the current value 
of and can therefore be ignored. 



25 



With regard to the prior probability density function for 
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the variance of the AR filter coefficients, the 
statistical analysis unit 21 could set this equal to some 
constant to imply that all variances are equally 
probable. However, this term can be used to introduce 
5 knowledge about what the variance of the AR filter 

coefficients is expected to be- In this embodiment, 
since variances are always positive, the statistical 
=y analysis unit 21 models this variance prior probability 

density function by an Inverse Gamma function having 

IJllO parameters a a and p a , i.e.: 

• f ^ 
m 

,2, o . ( a a) 

(I! ^ " ° PJ(a a ) 

111 
i = \ 

P At the beginning of the speech being processed, the 

15 statistical analysis unit 21 will not have much knowledge 

about the variance of the AR filter coefficients. 
Therefore, initially, the statistical analysis unit 21 
sets the variance a a 2 and the a and (3 parameters of the 
Inverse Gamma function to ensure that this probability 
20 density function is fairly flat and therefore non- 

informative. However, after the first frame of speech 
has been processed, these parameters can be set more 
accurately during the processing of the next frame of 
speech by using the parameter values calculated during 
25 the processing of the previous frame of speech. 



-1 



(17) 
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P(h\r) 

This term represents the prior probability density 
function for the channel model coefficients (h) and it 
allows the statistical analysis unit 21 to introduce 
5 knowledge about what values it expects these coefficients 

to take- As with the prior probability density function, 
for the AR filter coefficients, in this embodiment, this 

i~S 

r _ » 

=0 probability density function is modelled by a Gaussian 

u\ having an unknown variance (a h 2 ) and mean vector (u. h ) / 

1J! 10 i.e.: 

O PWr,o 2 h ,n h ) = (2no 2 h ) 2 exp 

zS : 
111 

t=i 

15 Again, by introducing these new variables, the prior 

density functions (p(a h ) and p(Mh) ) must be added to the 
numerator of equation (10). Again, the mean vector can 
initially be set to zero and after the first frame of 
speech has been processed and for all subsequent frames 

20 of speech being processed, the mean vector can be set to 

equal the mean vector obtained during the processing of 
the previous frame. Therefore, p(u h ) is also just a 
Dirac delta function located at the current value of ^ h 
and can be ignored. 

25 
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With regard to the prior probability density function for 
the variance of the channel filter coefficients, again, 
in this embodiment, this is modelled by an Inverse Gamma 
function having parameters a h and p h . Again, the variance 
(a h 2 ) and the a and p parameters of the Inverse Gamma 
function can be chosen initially so that these densities 
are non-informative so that they will have little effect 
on the subsequent processing of the initial frame. 

p(cr a 2 ) and p(cf/) 

These terms are the prior probability density functions 
for the process and measurement noise variances and 
again, these allow the statistical analysis unit 21 to 
introduce knowledge about what values it expects these 
noise variances will take- As with the other variances, 
in this embodiment, the statistical analysis unit 21 
models these by an Inverse Gamma function having 
parameters a e/ p e and a e , £ c respectively. Again, these 
variances and these Gamma function parameters can be set 
initially so that they are non-informative and will not 
appreciably affect the subsequent calculations for the 
initial frame. 

p(k) and p(r) 

These terms are the prior probability density functions 
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for the AR filter model order (k) and the channel model 
order (r) respectively. In this embodiment, these are 
modelled by a uniform distribution up to some maximum 
order. In this way, there is no prior bias on the number 
of coefficients in the models except that they can not 
exceed these predefined maximums. In this embodiment, 
the maximum AR filter model order (k) is thirty and the 
maximum channel model order (r) is one hundred and fifty. 

Therefore, inserting the relevant equations into the 
numerator of equation (10) gives the following joint 
probability density function which is proportional to 
P(a,k,h,r,a a 2 ,a h 2 ,a e 2 ,a e 2 , s(n) |y_(n) ) : 
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N 

,V 2 



(27ia e ) 2 exp 



-1 
2a 2 



(a(«) r a(«) ' 2h T V<g(n) + h T Y T Yh) 



N 

A 2. 



x (2tic^) 2 exp 



-1 
2a„ 



(i(«) r £(«) - 2a T SHn) + a T S T Sa) 



2x 2 



x (27ta a ) 2 exp 



-fa-U ) r (fl-U ) 



2a 



A 2 



x (2n<5 h ) 2 exp 



-{h-M h ) T {h-\L h ) 



lot 



exp 

P„r(a a ) 


-l 

°»P„ 


(°») (a ** 1> 

x exp 

P*r(a„) 


-1 

°*p* 


exp 

P,n<g 




x exp 

P c r(<g 




°% 





(19) 
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Gibbs Sampler 

In order to determine the form of this joint probability 
density function, the statistical analysis unit 21 "draws 
samples" from it. In this embodiment, since the joint 
probability density function to be sampled is a complex 
multivariate function, a Gibbs. sampler is used which 
breaks down the problem into one of drawing samples from 
J probability density functions of smaller dimensionality. 

In particular, the Gibbs sampler proceeds by drawing 



IV-10 random variates from conditional densities as follows: 

ii 

in first iteration 

in 



" 15 



e 



/ 2 1 , i , i , i i 2 l 2 1 2 1 , xo / v 2 1 

second iteration 
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etc . 



m 10 
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where <h°, r°, (a a 2 ) 0 , <a e 2 ) 0 , (a a 2 ) 0 , (^h 2 ) 0 / s(n)°) are 
initial values which may be obtained from the results of 
the statistical analysis of the previous frame of speech, 
or where there are no previous frames, can be set to 
appropriate values that will be known to those skilled in 
the art of speech processing. 

As those skilled in the art will appreciate, these 
conditional densities are obtained by inserting the 
current values for the given (or known) variables into 
the terms of the density function of equation (19). For 
the conditional density p(a,k|...) this results in: 
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which can be simplified to give: 
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which is in the form of a standard Gaussian distribution 
having the following covariance matrix: 
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The mean value of this Gaussian distribution can be 
determined by differentiating the exponent of equation 
(21) with respect to a and determining the value of a 
which makes the differential of the exponent equal to 
zero. This yields a mean value of: 



A sample can then be drawn from this standard Gaussian 
distribution to give a* (where g is the g th iteration of 
the Gibbs sampler) with the model order (k 9 ) being 
determined by a model order selection routine which will 
be described later. The drawing of a sample from this 
Gaussian distribution may be done by using a random 
number generator which generates a vector of random 
values which are uniformly distributed and then using a 
transformation of random variables using the covariance 
matrix and the mean value given in equations (22) and 
(23) to generate the sample. In this embodiment, 
however, a random number generator is used which 
generates random numbers from a Gaussian distribution 
having zero mean and a variance of one. This simplifies 
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the transformation process to one of a simple scaling 
using the covariance matrix given in equation (22) and 
shifting using the mean value given in equation (23). 
Since the techniques for drawing samples from Gaussian 
distributions are well known in the art of statistical 
analysis, a further description of them will not be given 
here- A more detailed description and explanation can be 
found in the book entitled "Numerical Recipes in C", by 
W. Press et al, Cambridge University Press, 1992 and in 
particular at chapter 7. 

As those skilled in the art will appreciate, however, 
before a sample can be drawn from this Gaussian 
distribution, estimates of the raw speech samples must be 
available so that the matrix S and the vector s(n) are 
known. The way in which these estimates of the raw 
speech samples are obtained in this embodiment will be 
described later. 
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A similar analysis for the conditional density p(h,r | . . . ) 
reveals that it also is a standard Gaussian distribution 
but having a covariance matrix and mean value given by: 
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from which a sample for h 9 can be drawn in the manner 
described above, with the channel model order (r^) being 
determined using the model order selection routine which 
will be described later. 

A similar analysis for the conditional density p(a e 2 | . . . ) 
shows that: 



PiplV -) « (<£) 2 exp 
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where : 



E = &(n) T &(n) - 2a T Sz(n) + a T S T S a 



which can be simplified to give: 




(26) 



which is also an Inverse Gamma distribution having the 
following parameters: 
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A sample is then drawn from this Inverse Gamma 
distribution by firstly generating a random number from 
a uniform distribution and then performing a 
transformation of random variables using the alpha and 
beta parameters given in equation (27), to give (a e 2 ) 9 . 

A similar analysis for the conditional density p(a e 2 |.„.) 
reveals that it also is an Inverse Gamma distribution 
having the following parameters: 



A sample is then drawn from this Inverse Gamma 
distribution in the manner described above to give (a e 2 ) 9 . 
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where : 



E* = q.{n) T q.(n) - 2h T Yq.(n) + h T Y T Y h 



A similar analysis for conditional density p(a a 2 |...) 
reveals that it too is an Inverse Gamma distribution 
having the following parameters: 
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A sample is then drawn from this Inverse Gamma 
distribution in the manner described above to give (a a 2 )9. 



Similarly, the conditional density p(a h 2 |...) is also an 
Inverse Gamma distribution but having the following 
parameters: 



A sample is then drawn from this Inverse Gamma 
distribution in the manner described above to give (a h 2 )*. 

As those skilled in the art will appreciate, the Gibbs 
sampler requires an initial transient period to converge 
to equilibrium (known as burn-in). Eventually, after L 
iterations, the sample (a L , k L , h L , r L , (a e 2 ) L , (a fr 2 ) L , 
(<? a 2 ) L / ( a h 2 ) L f s(n) L ) is considered to be a sample from the 
joint probability density function defined in equation 
(19). In this embodiment, the Gibbs sampler performs 
approximately one hundred and fifty (150) iterations on 
each frame of input speech and discards the samples from 
the first fifty iterations and uses the rest to give a 
picture (a set of histograms) of what the joint 
probability density function defined in equation (19) 
looks like. From these histograms, the set of AR 
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coefficients (a) which best represents the observed 
speech samples (y(n)) from the analogue to digital 
converter 17 are determined. The histograms are also 
used to determine appropriate values for the variances 
5 and channel model coefficients (h) which can be used as 

the initial values for the Gibbs sampler when it 
processes the next frame of speech. 

yi Model Order Selection 

III 10 As mentioned above, during the Gibbs iterations, the 

if: 

ill model order (k) of the AR filter and the model order (r) 

Si 

q of the channel filter are updated using a model order 

yj selection routine. In this embodiment, this is performed 

™% 

;='i using a technique derived from "Reversible jump Markov 

15 chain Monte Carlo computation", which is described in the 

paper entitled "Reversible jump Markov chain Monte Carlo 
Computation and Bayesian model determination" by Peter 
Green, Biometrika, vol 82, pp 711 to 732, 1995. 

20 Figure 4 is a flow chart which illustrates the processing 

steps performed during this model order selection routine 
for the AR filter model order (k). As shown, in step si, 
a new model order ( k 2 ) is proposed. In this embodiment, 
the new model order will normally be proposed as k 2 = ki 

25 ± 1, but occasionally it will be proposed as k 2 = k x ± 2 
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and very occasionally as k 2 = k x ± 3 etc. To achieve 
this, a sample is drawn from a discretised Laplacian 
density function centred on the current model order (k x ) 
and with the variance of this Laplacian density function 
being chosen a priori in accordance with the degree of 
sampling of the model order space that is required. 

The processing then proceeds to step s3 where a model 
order variable (MO) is set equal to: 



where the ratio term is the ratio of the conditional 
probability given in equation (21) evaluated for the 
current AR filter coefficients (a) drawn by the Gibbs 
sampler for the current model order (k^ and for the 
proposed new model order ( k 2 ) . If k 2 > k lf then the 
matrix S must first be resized and then a new sample 
must be drawn from the Gaussian distribution having the 
mean vector and covariance matrix defined by equations 
(22) and (23) (determined for the resized matrix S), to 
provide the AR filter coefficients (a <1:k2> ) for the new 
model order (k 2 ) . If k 2 < k t then all that is required is 
to delete the last (k x - k 2 ) samples of the a vector. If 
the ratio in equation (31) is greater than one, then this 




(31) 
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implies that the proposed model order (k 2 ) is better than 
the current model order whereas if it is less than one 
then this implies that the current model order is better 
than the proposed model order. However, since 

5 occasionally this will not be the case, rather than 

deciding whether or not to accept the proposed model 
order by comparing the model order variable (MO) with a 
;]£ fixed threshold of one, in this embodiment, the model 

!"=; order variable (MO) is compared, in step s5, with a 

is-. 

ji* 10 random number which lies between zero and one. If the 

in model order variable (MO) is greater than this random 

i=-x number, then the processing proceeds to step s7 where the 

! fl 

Hi model order is set to the proposed model order (k 2 ) and 

j=% a count associated with the value of k 2 is incremented. 

J ~15 If, on the other hand, the model order variable (MO) is 

smaller than the random number, then the processing 
proceeds to step s9 where the current model order is 
maintained and a count associated with the value of the 
current model order (k x ) is incremented. The processing 
20 then ends. 

This model order selection routine is carried out for 
both the model order of the AR filter model and for the 
model order of the channel filter model. This routine 
25 may be carried out at each Gibbs iteration. However, 
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this is not essential. Therefore, in this embodiment, 
this model order updating routine is only carried out 
every third Gibbs iteration. 

5 Simulation Smoother 

As mentioned above, in order to be able to draw samples 
using the Gibbs sampler, estimates of the raw speech 
samples are required to generate s_(n), S and Y which are 

'■sir 
— I 

*:J; used in the Gibbs calculations. These could be obtained 

!%! 1 0 from the conditional probability density function 

si | 
- '-. 

I~ p( s.( n) | . . . ) . However, this is not done in this 

;.L embodiment because of the high dimensionality of S(n) . 

1=1 

r; Therefore, in this embodiment, a different technique is 

Ul 

used to provide the necessary estimates of the raw speech 

5 -. 

3= 15 samples. In particular, in this embodiment, a 

"Simulation Smoother" is used to provide these estimates. 
This Simulation Smoother was proposed by Piet de Jong in 
the paper entitled "The Simulation Smoother for Time 
Series Models", Biometrika (1995), vol 82,2, pages 339 to 

20 350. As those skilled in the art will appreciate, the 

Simulation Smoother is run before the Gibbs Sampler. It 
is also run again during the Gibbs iterations in order to 
update the estimates of the raw speech samples. In this 
embodiment, the Simulation Smoother is run every fourth 

25 Gibbs iteration. 
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In order to run the Simulation Smoother, the model 
equations defined above in equations (4) and (6) must be 
written in "state space" format as follows: 



Sin) = A.£(n-l) + e(n) 
An) = h r .£(n-\) + e(/i) 
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With this state space representation, the dimensionality 
of the raw speech vectors (s(n) ) and the process noise 
vectors (e(n)) do not need to be Nxl but only have to be 
as large as the greater of the model orders - k and r. 
Typically, the channel model order (r) will be larger 
than the AR filter model order (k). Hence, the vector of 
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raw speech samples (s(n)) and the vector of process noise 
(e(n)) only need to be rxl and hence the dimensionality 
of the matrix A only needs to be rxr. 



The Simulation Smoother involves two stages - a first 
stage in which a Kalman filter is run on the speech 
samples in the current frame and then a second stage in 
which a "smoothing" filter is run on the speech samples 
in the current frame using data obtained from the Kalman 

0~i 10 filter stage. Figure 5 is a flow chart illustrating the 

HI 

'&\ processing steps performed by the Simulation Smoother- 

i 

: =_ As shown, in step s21, the system initialises a time 

UI variable t to equal one. During the Kalman filter stage, 

this time variable is run from t = 1 to N in order to 
Nl5 process the N speech samples in the current frame being 

processed in time sequential order. After step s21, the 
processing then proceeds to step s23, where the following 
Kalman filter equations are computed for the current 
speech sample (y(t) ) being processed: 

HO = y(t) - h T i.{t) 
d{t) = h T P{t)h + ol 
kit) = {AP(t)li).d(tr l 

(33) 

fif+l) = Am + k f (0M0 

L(t) = A - k f {t).h T 
P(t+\) = AP(t)L{t) T + a 2 e .I 
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where the initial vector of raw speech samples (s(l)) 
includes raw speech samples obtained from the processing 
of the previous frame (or if there are no previous frames 
then s(i) is set equal to zero for i < 1); P(l) is the 
5 variance of s(l) (which can be obtained from the previous 

frame or initially can be set to a e 2 ); h is the current 
set of channel model coefficients which can be obtained 
from the processing of the previous frame (or if there 
are no previous frames then the elements of h can be set 
Hi 10 to their expected values - zero); y(t) is the current 

if; 

{l\ speech sample of the current frame being processed and I 

J =1 is the identity matrix. The processing then proceeds to 

Y\ step s25 where the scalar values w(t) and d(t) are stored 

•=I together with the rxr matrix L(t) (or alternatively the 

i_ ~15 Kalman filter gain vector k f (t) could be stored from 

which L(t) can be generated) . The processing then 
proceeds to step s27 where the system determines whether 
or not all the speech samples in the current frame have 
been processed. If they have not, then the processing 
20 proceeds to step s2 9 where the time variable t is 

incremented by one so that the next sample in the current 
frame will be processed in the same way. Once all N 
samples in the current frame have been processed in this 
way and the corresponding values stored, the first stage 
2 5 of the Simulation Smoother is complete. 
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The processing then proceeds to step s31 where the second 
stage of the Simulation Smoother is started in which the 
smoothing filter processes the speech samples in the 
current frame in reverse sequential order- As shown, in 
5 step s31 the system runs the following set of smoothing 

filter equations on the current speech sample being 
processed together with the stored Kalman filter 
variables computed for the current speech sample being 
processed: 

10 2 2 

n (t)~N(0,C(t)) 
V(t) = o 2 e U(t)L(t) 

1 c(t-i) = hd(ty l w(o + L{t) T t{t) - v(0 T C(ty l n(0 

; (34) 

: u{t-\) = hd{ty x h T + L{tfu{t)L{t) + v(t) T C(ty l v(t) 

15 

Ht) = o 2 e c{t) + n(0 where £(/) = [e(t) e(t-\) e(t-2) ... e(t-r+\)] T 
l(t) = Al(t-\) + £(/) where l(t) = [s(t) s(t-\) s{t-2) ... s(t-r+\)f 

and e(t) = [e(t) 0 0 ... 0 f 

where n('t) is a sample drawn from a Gaussian distribution 
20 having zero mean and covariance matrix C(t); the initial 

vector r(t=N) and the initial matrix U(t=N) are both set 
to zero; and s(0) is obtained from the processing of the 
previous frame (or if there are no previous frames can be 
set equal to zero) . The processing then proceeds to step 

2 5 s3 3 where the estimate of the process noise <e(t)) for 
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the current speech sample being processed and the 
estimate of the raw speech sample (s(t)) for the current 
speech sample being processed are stored. The processing 
then proceeds to step s35 where the system determines 
5 whether or not all the speech samples in the current 

frame have been processed. If they have not, then the 
processing proceeds to step s3 7 where the time variable 
=y t is decremented by one so that the previous sample in 

yi the current frame will be processed in the same way. 

LnlO Once all N samples in the current frame have been 

IJi processed in this way and the corresponding process noise 

if 

~Q) and raw speech samples have been stored, the second stage 

■r i 

hi of the Simulation Smoother is complete and an estimate of 

Q ii(n) will have been generated. 

1 15 

As shown in equations ( 4 ) and ( 8 ) , the matrix S and the 
matrix Y require raw speech samples s(n-N-l) to s(n-N- 
k+1) and s(n-N-l) to s(n-N-r+l) respectively in addition 
to those in s(n) . These additional raw speech samples 

20 can be obtained either from the processing of the 

previous frame of speech or if there are no previous 
frames, they can be set to zero. With these estimates of 
raw speech samples, the Gibbs sampler can be run to draw 
samples from the above described probability density 

25 functions. 
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Statistical Analysis Unit - Operation 

A description has been given above of the theory 
underlying the statistical analysis unit 21. A 
description will now be given with reference to Figures 
5 6 to 8 of the operation of the statistical analysis unit 

21 that is used in the embodiment. 

su Figure 6 is a block diagram illustrating the principal 

m 

ill components of the statistical analysis unit 21 of this 

till 0 embodiment. As shown, it comprises the above described 

-n 

lit Gibbs sampler 41, Simulation Smoother 43 (including the 

!3 Kalman filter 43-1 and smoothing filter 43-2) and model 

hi order selector 45. It also comprises a memory 47 which 

13 receives the speech samples of the current frame to be 

15 processed, a data analysis unit 49 which processes the 

data generated by the Gibbs sampler 41 and the model 
order selector 45 and a controller 50 which controls the 
operation of the statistical analysis unit 21. 

20 As shown in Figure 6, the memory 47 includes a non 

volatile memory area 47-1 and a working memory area 47-2. 
The non volatile memory 47-1 is used to store the joint 
probability density function given in equation (19) above 
and the equations for the variances and mean values and 

2 5 the equations for the Inverse Gamma parameters given 
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above in equations (22) to (24) and (27) to (30) for the 
above mentioned conditional probability density functions 
for use by the Gibbs sampler 41. The non volatile memory 
47-1 also stores the Kalman filter equations given above 
5 in equation (33) and the smoothing filter equations given 

above in equation 34 for use by the Simulation Smoother 
43. 

• f\ 

'Biff 

Figure 7 is a schematic diagram illustrating the 

*/tL 0 parameter values that are stored in the working memory 

ill area (RAM) 47-2. As shown, the RAM includes a store 51 

ui for storing the speech samples y f (1) to y f (N) output by 

111 

Ul the analogue to digital converter 17 for the current 

\~i 

1=1 frame (f) being processed. As mentioned above, these 

15 speech samples are used in both the Gibbs sampler 41 and 

the Simulation Smoother 43. The RAM 47-2 also includes 
a store 53 for storing the initial estimates of the model 
parameters (g=0) and the M samples (g = 1 to M) of each 
parameter drawn from the above described conditional 
20 probability density functions by the Gibbs sampler 41 for 

the current frame being processed. As mentioned above, 
in this embodiment, M is 100 since the Gibbs sampler 41 
performs 150 iterations on each frame of input speech 
with the first fifty samples being discarded. The RAM 
25 47-2 also includes a store 55 for storing W(t), d(t) and 



L(t) for t = 1 to N which are calculated during the 
processing of the speech samples in the current frame of 
speech by the above described Kalman filter 43-1. The 
RAM 47-2 also includes a store 57 for storing the 
estimates of the raw speech samples (s f (t)) and the 
estimates of the process noise (e f (t) ) generated by the 
smoothing filter 43-2, as discussed above. The RAM 47-2 
also includes a store 59 for storing the model order 
counts which are generated by the model order selector 45 
when the model orders for the AR filter model and the 
channel model are updated. 

Figure 8 is a flow diagram illustrating the control 
program used by the controller 50, in this embodiment, to 
control the processing operations of the statistical 
analysis unit 21. As shown, in step s41, the controller 
50 retrieves the next frame of speech samples to be 
processed from the buffer 19 and stores them in the 
memory store 51. The processing then proceeds to step 
s4 3 where initial estimates for the channel model, raw 
speech samples and the process noise and measurement 
noise statistics are set and stored in the store 53. 
These initial estimates are either set to be the values 
obtained during the processing of the previous frame of 
speech or, where there are no previous frames of speech, 
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are set to their expected values (which may be zero). 
The processing then proceeds to step s45 where the 
Simulation Smoother 43 is activated so as to provide an 
estimate of the raw speech samples in the manner 
5 described above. The processing then proceeds to step s47 

where one iteration of the Gibbs sampler 41 is run in 
order to update the channel model, speech model and the 

;Jf process and measurement noise statistics using the raw 

m 

ji? speech samples obtained in step s45. These updated 

parameter values are then stored in the memory store 53. 



=f The processing then proceeds to step s49 where the 

y controller 50 determines whether or not to update the 

=^ model orders of the AR filter model and the channel 

15 model. As mentioned above, in this embodiment, these 

model orders are updated every third Gibbs iteration. If 
the model orders are to be updated, then the processing 
proceeds to step s51 where the model order selector 45 is 
used to update the model orders of the AR filter model 
20 and the channel model in the manner described above. If 

at step s49 the controller 50 determines that the model 
orders are not to be updated, then the processing skips 
step s51 and the processing proceeds to step s53. At 
step s53, the controller 50 determines whether or not to 
2 5 perform another Gibbs iteration. If another iteration is 
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to be performed, then the processing proceeds to decision 
block s55 where the controller 50 decides whether or not 
to update the estimates of the raw speech samples (s(t)). 
If the raw speech samples are not to be updated, then the 
5 processing returns to step s47 where the next Gibbs 

iteration is run. 

1=1 

K U As mentioned above, in this embodiment, the Simulation 

]i} Smoother 43 is run every fourth Gibbs iteration in order 

410 to update the raw speech samples. Therefore, if the 

Ml controller 50 determines, in step s55 that there has been 

four Gibbs iterations since the last time the speech 
Ul samples were updated, then the processing returns to step 

1=1 s45 where the Simulation Smoother is run again to provide 

15 new estimates of the raw speech samples (s(t)). Once the 

controller 50 has determined that the required 150 Gibbs 
iterations have been performed, the controller 50 causes 
the processing to proceed to step s57 where the data 
analysis unit 49 analyses the model order counts 
20 generated by the model order selector 45 to determine the 

model orders for the AR filter model and the channel 
model which best represents the current frame of speech 
being processed. The processing then proceeds to step 
s59 where the data analysis unit 49 analyses the samples 
25 drawn from the conditional densities by the Gibbs sampler 



41 to determine the AR filter coefficients (a) , the 
channel model coefficients (h) , the variances of these 
coefficients and the process and measurement noise 
variances which best represent the current frame of 
speech being processed. The processing then proceeds to 
step s61 where the controller 50 determines whether or 
not there is any further speech to be processed. If 
there is more speech to be processed, then processing 
returns to step S41 and the above process is repeated for 
the next frame of speech. Once all the speech has been 
processed in this way, the processing ends. 

Data Analysis unit 

A more detailed description of the data analysis unit 49 
will now be given with reference to Figure 9. As 
mentioned above, the data analysis unit 49 initially 
determines, in step s57, the model orders for both the AR 
filter model and the channel model which best represents 
the current frame of speech being processed. It does 
this using the counts that have been generated by the 
model order selector 45 when it was run in step s51. 
These counts are stored in the store 59 of the RAM 47-2. 
In this embodiment, in determining the best model orders, 
the data analysis unit 49 identifies the model order 
having the highest count. Figure 9a is an exemplary 
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histogram which illustrates the distribution of counts 
that is generated for the model order (k) of the AR 
filter model. Therefore, in this example, the data 
analysis unit 49 would set the best model order of the AR 
5 filter model as five- The data analysis unit 49 performs 

a similar analysis of the counts generated for the model 
order (r) of the channel model to determine the best 
model order for the channel model. 



^io Once the data analysis unit 49 has determined the best 

Wi model orders (k and r), it then analyses the samples 

v. 

j-( generated by the Gibbs sampler 41 which are stored in the 

^ store 53 of the RAM 47-2, in order to determine parameter 

^ values that are most representative of those samples. 

15 It does this by determining a histogram for each of the 

parameters from which it determines the most 
representative parameter value. To generate the 

histogram, the data analysis unit 49 determines the 
maximum and minimum sample value which was drawn by the 
2 0 Gibbs sampler and then divides the range of parameter 

values between this minimum and maximum value into a 
predetermined number of sub-ranges or bins. The data 
analysis unit 49 then assigns each of the sample values 
into the appropriate bins and counts how many samples 
2 5 are allocated to each bin. It then uses these counts to 
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calculate a weighted average of the samples (with the 
weighting used for each sample depending on the count for 
the corresponding bin), to determine the most 
representative parameter value (known as the minimum mean 



example histogram which is generated for the variance 
(a e 2 ) of the process noise, from which the data analysis 
unit 4 9 determines that the variance representative of 
the sample is 0.3149. 

In determining the AR filter coefficients (a A for i = i 
to k) , the data analysis unit 49 determines and analyses 
a histogram of the samples for each coefficient 
independently. Figure 9c shows an exemplary histogram 
obtained for the third AR filter coefficient (a 3 ), from 
which the data analysis unit 49 determines that the 
coefficient representative of the samples is -0.4977. 

In this embodiment, the data analysis unit 49 outputs the 
AR coefficients (a) and the AR filter model order (k) . 
The AR filter coefficients (a) are output to both the 
speech recognition unit 25 and the speech detection unit 
61, whereas the AR filter model order (k) is only output 
to the speech detection unit 61. These parameter values 
(and the remaining parameter values determined by the 



square estimate (MMSE ) ) . 



Figure 9b illustrates an 
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data analysis unit 49) are also stored in the RAM 47-2 
for use during the processing of the next frame of 
speech. As mentioned above, the speech detection unit 61 
compares the AR filter model order (k) and the AR filter 
5 coefficient values with appropriate threshold values, and 

determines that speech is present within the input signal 
;= , when the AR filter model order and the AR filter 

:J( coefficient values exceed these threshold values. When 

ji: the speech detection unit 61 detects the presence of 

;%0 speech, it outputs an appropriate control signal to the 

speech recognition unit 25, which causes it to start 
j=f processing the AR coefficients it receives from the 

]ti statistical analysis unit 21. Similarly, when the speech 

; =i detection unit 61 detects the end of speech, it outputs 

1== 

15 an appropriate control signal to the speech recognition 

unit 25 which causes it to stop processing the AR 
coefficients it receives from the statistical analysis 
unit 21. 

2 0 As those skilled in the art will appreciate, a technique 

has been described above which employs a statistical 
analysis to determine AR coefficients and AR model order 
which are used by a speech detection unit to detect the 
presence of speech within an input signal. The technique 

2 5 is more robust and accurate than prior art techniques 
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which compare the energy of the input signal with some 



techniques described above are also more robust and 
accurate than prior art techniques which employ maximum 
likelihood estimators to determine these coefficients. 
This is because the statistical analysis of each frame 
uses knowledge obtained from the processing of the 
previous frame. In addition, with the analysis performed 
above, the model order for the AR filter model is not 
assumed to be constant and can vary from frame to frame. 
In this way, the optimum number of AR filter coefficients 
can be used to represent the speech within each frame. 
As a result, the AR filter coefficients output by the 
statistical analysis unit 21 will more accurately 
represent the corresponding input speech. Further still, 
since the underlying process model that is used separates 
the speech source from the channel, the AR filter 
coefficients that are determined will be more 
representative of the actual speech and will be less 
likely to include distortive effects of the channel. 
Further still, since variance information is available 
for each of the parameters, this provides an indication 
of the confidence of each of the parameter estimates. 
This is in contrast to maximum likelihood and least 
squares approaches, such as linear prediction analysis, 



threshold value. 



Further, the statistical analysis 
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where point estimates of the parameter values are 
determined. 

Alternative Embodiments 

5 In the above embodiment, the statistical analysis unit 

was used as a pre-processor for a speech recognition 
system in order to generate AR coefficients 
Jf representative of the input speech- The statistical 

i! analysis unit was also used to determine the AR filter 

=10 model order which was used together with the AR 

" coefficients by a speech detection unit to detect the 

=j presence of speech within the input signal. As those 

f 1 skilled in the art will appreciate, since both the model 

=^ order and the values of the AR coefficients will vary 

15 depending on whether or not there is speech present 

within the input signal, the speech detection unit can 
detect the presence of speech using only the AR filter 
model order or only the AR coefficient values. However, 
in the preferred embodiment, both the model order and the 
20 AR coefficient values are used, since this allows a more 

accurate speech detection to be performed. For example, 
for speech sounds where there is a weak correlation 
between adjacent speech samples (such as fricative 
sounds), if only the AR coefficient values are used, then 
25 the presence of such fricative sounds may be missed since 



all the AR filter coefficients may have small values 
below the corresponding threshold values. Nonetheless, 
with such fricative sounds, the model order is likely to 
exceed its threshold value, in which case the speech 
detection unit can still reliably detect the speech. 

In the above embodiments, a speech detection system was 
described in use together with a speech recognition 
system. As those skilled in the art will appreciate, the 
speech detection system described above may be used in 
any speech processing system to control the initiation 
and termination of the speech processing operation. For 
example, it can be used in a speaker verification system 
or in a speech transmission system in order to control 
the verification process and the transmission process 
respectively . 

In the above embodiment, the statistical analysis unit 
was used effectively as a "preprocessor" for both the 
speech recognition unit and the speech detection unit. 
As those skilled in the art will appreciate, in an 
alternative embodiment, a separate preprocessor may be 
provided as the front end to the speech recognition unit. 
In this case, the statistical analysis unit would only be 
used to provide information to the speech detection unit. 



However, such separate parameter! sation of the input 
speech for the speech recognition unit is not preferred 
because of the additional processing overhead involved. 

In the above embodiment, a speech recognition system was 
used which used the AR filter coefficients output by the 
statistical analysis unit. In embodiments where the 
speech recognition unit does not use AR filter 
coefficients but uses other spectral based coefficients 
(such as cepstral coefficients), an appropriate 
coefficient converter may be used to convert the AR 
coefficients into the appropriate coefficients for use by 
the speech recognition unit. 

In the above embodiments, Gaussian and Inverse Gamma 
distributions were used to model the various prior 
probability density functions of equation (19). As those 
skilled in the art of statistical analysis will 
appreciate, the reason these distributions were chosen is 
that they are conjugate to one another. This means that 
each of the conditional probability density functions 
which are used in the Gibbs sampler will also either be 
Gaussian or Inverse Gamma. This therefore simplifies the 
task of drawing samples from the conditional probability 
densities. However, this is not essential. The noise 
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probability density functions could be modelled by 
Laplacian or student-t distributions rather than Gaussian 



functions for the variances may be modelled by a 
distribution other than the Inverse Gamma distribution. 
For example, they can be modelled by a Rayleigh 
distribution or some other distribution which is always 
positive- However, the use of probability density 
functions that are not conjugate will result in increased 
complexity in drawing samples from the conditional 
densities by the Gibbs sampler. 

Additionally, whilst the Gibbs sampler was used to draw 
samples from the probability density function given in 
equation (19), other sampling algorithms could be used. 
For example the Metropolis-Hastings algorithm (which is 
reviewed together with other techniques in a paper 
entitled "Probabilistic inference using Markov chain 
Monte Carlo methods" by R. Neal, Technical Report CRG- 
TR-93-1, Department of Computer Science, University of 
Toronto, 1993) may be used to sample this probability 
density . 

In the above embodiment, a Simulation Smoother was used 
to generate estimates for the raw speech samples. This 



distributions . 



Similarly, the probability density 
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Simulation Smoother included a Kalman filter stage and a 
smoothing filter stage in order to generate the estimates 
of the raw speech samples. In an alternative embodiment, 
the smoothing filter stage may be omitted, since the 
Kalman filter stage generates estimates of the raw speech 
(see equation (33)). However, these raw speech samples 
were ignored, since the speech samples generated by the 
smoothing filter are considered to be more accurate and 
robust- This is because the Kalman filter essentially 
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ill 10 generates a point estimate of the speech samples from the 



3i joint probability density function p( s ( n) | a, k, a e 2 ) , 

HI whereas the Simulation Smoother draws a sample from this 

ri probability density function. 

15 In the above embodiment, a Simulation Smoother was used 

in order to generate estimates of the raw speech samples. 
It is possible to avoid having to estimate the raw speech 
samples by treating them as "nuisance parameters" and 
integrating them out of equation (19). However, this is 

20 not preferred, since the resulting integral will have a 

much more complex form than the Gaussian and Inverse 
Gamma mixture defined in equation (19). This in turn will 
result in more complex conditional probabilities 
corresponding to equations (20) to (30). In a similar 

25 way, the other nuisance parameters (such as the 



coefficient variances or any of the Inverse Gamma, alpha 
and beta parameters) may be integrated out as well. 
However, again this is not preferred, since it increases 
the complexity of the density function to be sampled 
using the Gibbs sampler- The technique of integrating 
out nuisance parameters is well known in the field of 
statistical analysis and will not be described further 
here . 

In the above embodiment, the data analysis unit analysed 
the samples drawn by the Gibbs sampler by determining a 
histogram for each of the model parameters and then 
determining the value of the model parameter using a 
weighted average of the samples drawn by the Gibbs 
sampler with the weighting being dependent upon the 
number of samples in the corresponding bin- In an 
alterative embodiment, the value of the model parameter 
may be determined from the histogram as being the value 
of the model parameter having the highest count. 
Alternatively, a predetermined curve (such as a bell 
curve) could be fitted to the histogram in order to 
identify the maximum which best fits the histogram. 

In the above embodiment, the statistical analysis unit 
modelled the underlying speech production process with a 
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separate speech source model (AR filter) and a channel 
model. Whilst this is the preferred model structure, the 
underlying speech production process may be modelled 
without the channel model. In this case, there is no 
need to estimate the values of the raw speech samples 
using a Kalman filter or the like, although this can 
still be done. However, such a model of the underlying 
speech production process is not preferred, since the 
speech model will inevitably represent aspects of the 
channel as well as the speech. Further, although the 
statistical analysis unit described above ran a model 
order selection routine in order to allow the model 
orders of the AR filter model and the channel model to 
vary, this is not essential. 

In the above embodiments, the speech that was processed 
was received from a user via a microphone. As those 
skilled in the art will appreciate, the speech may be 
received from a telephone line or may have been stored on 
a recording medium. In this case, the channel model will 
compensate for this so that the AR filter coefficients 
representative of the actual speech that has been spoken 
should not be significantly affected. 

In the above embodiments, the speech generation process 



was modelled as an auto-regressive (AR) process and the 
channel was modelled as a moving average (MA) process. 
As those skilled in the art will appreciate, other signal 
models may be used- However, these models are preferred 
because it has been found that they suitably represent 
the speech source and the channel they are intended to 
model . 

In the above embodiments, during the running of the model 
order selection routine, a new model order was proposed 
by drawing a random variable from a predetermined 
Laplacian distribution function. As those skilled in 
the art will appreciate, other techniques may be used. 
For example the new model order may be proposed in a 
deterministic way (ie under predetermined rules), 
provided that the model order space is sufficiently 
sampled. 



